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ABSTRACT 

In order to determine tha effectiveness of 
multidimansional scaling (IfflS) in racovaring the dimansionality of a 
set of dichotomously-scored items, data were simulatad in one, two, 
and three dimensions for a variety of correlations with the 
underlying latent trait « Similarity matrices wara constructad from 
these data using thraa margin^sansitive and three margin-fra.s 
coafficientf and used as input to ^S, Strass (SI), SI by dimansion 
plots r and plots of the scalad items wara aKamined to detarmine the 
effect of varying the magnitude and pattarn of correlations. The 
results suggested that itams with similar patterns of correlations 
tend to cluster togethar, that distanca from tha cantar of a clustar 
is a function of the amount of random arror in tha item, and that as 
the number of latent traits underlying the data increases, the ~ 
dimansionality of tha representational spaca incraasesa Clustar 
analysis using HDS coordinates is suggested to isolate homoganaous 
sats of items, wharaas considaration of tha SI coefficient is 
racommendad to datermina tha nmnber of latant traits in tha data, 
(Author) 
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ABSTRACT 

In fc© determine the ef f eativenese of multidlmenaicinal 

mmmlXng <HDS> in areaave^lng the diAenalanallty q£ a aet of 
diahotQBiaualy-sQOred itema, date wa^^e almulatad in one, twa^ and 
ths'aa dimenaiana £ar a variety a£ aarrelatiane ^ith the 
underlying latent trait. Simiiarity matriaea were aanetruated 
fran theee data ualng. three nargin-aenaitive. and three margin^ 
free eaefflaianta and uaed ae input to MDS. Streaa <M%> ^ B± by 
dijienaian plata^ and plata m£ the aaaled itema were examined to 
determine the effeat a£ varying the magnitude and pattern a£ 
aarrelatiana. The raaulta auggeated that itema with aimilar 
patterns af aorrelatiana tend ta aluater together ^ that diatanae 
fran the aenter af a aluater ia a funatian af the amount af 
random errar in the item* and that mm the number of latent traits 
underlying the data inareaaea, the dimenaianality m£ the 
repreaentatianal epaae inareaaea^ Cluater analyaia uaing HDS 
caardinatea ia auggeated ta iaalata hamageneaua seta q£ itema* 
whereaa aanaideratian o£ the Si aaeffiaient ia reaammended ta 
determine the number af latent traita in the data« 



Datarmining what a met of itema raally mQaauras ramaina a problam 
in aduQatlonal pay^halagy today juat as it waa fifty yaara a^a^ Tha 
problaa is that diffaring mathadalogiaa far aHamlning tha atruotura af 
an itan sat may yiald dlff€*Mnt raaulta, baaauaa thair aaauiiptiona and 
tachniquaa dltffar. Thia ia not naMaaarily a drawback ainoa this may 
aaptura tha riahnaaa of tha data« Tha purpeaa of thia papar ia to 
aKamiria an altarnativa approach to datarmining tha atru^tura of a aat 
of itaaa uaing a eaabinatian of multidimanaianal aaaling and aluatar 
analyais. 

Inveatigatiana inta tha atruetura of a sat ©f itama fraquantly 
Inveiva ona or mora of thraa quaatlonai whathar tha Itama ara parvadad 
by a #insla undarlylng trait i how many traits parvada tha data if tha 
itama ara not unidimanaional and whioh itama ara ralatad to whioh 
traitai or whioh itama ara suffioiantiy similar that thay could ba aaid 
to Gonatituta a unidimanaional aat of itama. Aasaaaing sats of i tarns 
for unidimanaional Ity or forming unidimanaional aats of itama has 
baaoma ineraaaingly important ainoa tha davalopmant and implamantation 
of item raaponaa thaory <IHT> baoauaa moat aoallng praaaduras currantly 
raquira looal Indapandanoa and unidlmanslonality * No dafinltiva 
mathodology haa yat ^mmn advanaad for taatlng for unldimanalonality and 
Lord <lsao> haa pointad out tha naad for suah a taat. To tha aHtent 
that tha aaaumptlons of tha IRT madals ara not mat, paramatar astimatas 
will ba inaQQurata. 

Nultidimanaional IHT modals hava baan proposad and/or inveatigated 
by a numbar of raaaarohara inoluding Bock and Aitkin <1%B1> and 
KoKinlay and Raokasa Cldsaa, isa^b, 19a3> . Howavar* in ordar to 
astimata paramatars of aush modala, it ia naoaasary to know how many 
latant traits undarlla tha data. Thua, taohniquas for datarraining tha 
numbar of dimanaiona undarlying tha data ara naoaaaary for multivariate 
aa wall as unlvarlata modala* Glaaaioal approaohas to taat thaory hava 
raquirad aaaaasmant of dimanaionality to oonfirm tha appropriatanasa of 
providing a slngla tast soora for a subaat of itama. Contant validity 
atudiaa hava raquirad that tha tast davalopar detarmine whethar itama 
whioh ara aaaumad to share a oommon trait do, in faot, ahara the same 
trait. Thus, while tha problama of dimenaionality asaaaamant have 
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Cttealvad mumh mt,t.mntLM in raaent years, ^ho need for ^eohnlques to 
parmi'b dat^emindLng what, a aet, m£ It^ams mm^muTmm ia na^ naw and ia 
mmlmly ralat^ad ta IRT. 



Claaaiaal prlnaipal aampanant.a €PCA> and fastar analyaia (Gnanadar^- 
alkanp 1977> hava fraquantly baan uaad t»a d^at^arniina t.hm undarlylng 
at.ruat.ura a£ t.ha data* In tha FCA madal, individual ^^mmpmnmmm ara 
a^auaad ta lia in a' apaaa whaaa aK@a <nat. naaaaaarily art^haganal) ara 
famad by tha itana^ Tha gaal a£ PCA is ta farm art^haganal linear 
aQmbinat.iana af tha it^aaa suah t.hat eaah linaar aambinatiian aaaaunta 
far a maximum amaunt af t.ha residual ^m^Lmnmm left unaKplainad by 
praviaua linaar cambinatiana. Tha first, prinaipal aampanant is 
assantially a laaat.-squaras ragraaaian af a lat^ant trait, an the itam 
raapansas^ it. aaaaunt^a far mora varianaa t.han any athar possibla 
abmbinatian af itama. By suaaassivaly asc^raating linaar aambinat^ians 
whiah maximally aaaount. far raaidual ^mTLmnamp arthaganal axas ara 
farmed p If t.ha mat.riK af int.arit.am aorralatians is lass than full rank 
o£' if it. aan ba datarminad t.hat. only t.ha first. ^ £< ^3 prinaipal 
aompohant^a are needed ta aaaaunt. for a auff iaiant proportion af the 
variaAaa# a spaaa af smaller dimanaianalit.y is aanaiderad auff iaiant. ta 
aantain the struat.ura rapraaanted by the original data. The prinaipal 
aQmp0nanta whiah aanati-tuta t.ha axes af tpha aalut.ian may be interpreted 
mm repreaentingr unsaan faa'tara underlying tha data. If a aingla 
p^^inaipal aampanen^ appears ta aaaaun^ far a suffiaiant partian af the 
varianaa, than t.ha sat af . itama ia aaid t.a ba unidimenaianal . 

In t.ha faatar analytiaal modal ^ aaoh abaerved variable is assumed 
ta be aomprisad Qf a waight.ed linaar aambinat^ian a£ latent Ci.a., 
unmaaaurable) variables oallad aamman faa^ara whiah aaaount far the 
aamman varianaa amang the obsarvad variables aa well aa a unique latent 
variable <aallad t.ha spaaifia faa-tar) whiah aaaount.a far the varianaa 
uneHplainad by t.he aamman faatars. In this models the abaerved 
reapansa is itm^T^mmmmd^ an the latent, raapanaea. The PCA and faatar 
analysis madala are very similar in their , appraaahes to tha reduatian 
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o£ t,hm dlAenslQnsli'by af ^hm original dat.a mpmmm and will b@ regarded 
aa aasant^ially t.ha aana haifa« 

Alt^hough fao^Qir ^nalytiaai ^aehniquea would appear to be idi^aily 
auit^ad £©r datarmining dimaneianalit.y , aovaral problana aseur in fchair 
uaan Firat, t^has^e ia no sat, ^ritLarion for datarmining how many faatora 

principal €omponant,a to axtra^t in a givan analysis* Saoondly, 
£aat.Qr analysia raquiraa tha apaaifio aaaumption t^hat tha obaarvad 
variablaa rapraaant linaar coabinationa o£ tha latant variablaa. This 
praaan^a problaaa tf©r diehotomi^ad variablaa. If a faetor analyaia ia 
aondUQtad an a matriH of phi oorralatian aoaffiaianta Cwhiah implicitly 
aaauaa that tha diatributiona undarlying tha variablaa ara trua diahot- 
omiaa> ^ tha modal ia niaapaaii' iad ainca tha ragraaaion of a binary 
variabla on a cQntinuoua variabla ia not linaar CMcDonald and Ahlawat, 
19741 Mialavy^ lSdS> , Sinca tha ranga of tha valuaa of phi 
caafficianta are m££mm%md. by tha pairwiaa diatributiona ©£ the means of 
the binary variablaa Ci.e-, the claasical itam dif f icultiea> , tha 
magnitude of tha phi coaff iciant ia affaotad by tha itam diffleulty aa 
wall aa tha atrength of tha ralationahipa among the variablaa, Aa a 
reaultp extranaaus factors appear which ara relatad to tha difficulties 
of the items and not to any other source of common variance among the 
items # This clouds the iaaue of tha true dimensionality of tha item 
set • 

Ona potential solution to tha problems listed above ia to factor 
analyse a matrix of tatrachoric correlation coaf f icienta, which mmmumm 
that bivariate normal diatributiona underly tha pairwiaa distributions 
of dichotomized raaponses^ While thla aolvea tha problem of nonlinear 
ragraaaiona^ it bringa about other problema. First, if the aaaumption 
of bivariata normality is not mat^ the tatrachoric correlations will 
not raflect tha true ralationshipa among tha variablaa « Second ^ 
tatrachoric correlation coefficients ara not directly estimable. 
Simple approximation formulaa may ba found in many texta Csae^ a.g.. 
Lord and Novicte , 19Sa> p but thaae are accurate only in tha neighborhood 
of r 3 O.S, Mora complax estimation proceduras using Qauss-Harmite 
quadrature or Mewton-Haphson iterations have been suggested by 
Castallan <im&> , Kirk <iS73>, Divgl C1979>, and others- While these 
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tpeahniquu Improve thm maaujtmay m£ -thm mmmpu^m^Lonm , ^hmy omn hmmmmm 
unm'tm^lm whan onm or moitm amXX praport.iana of t.he paix*wiaa it^am 
raapanae t,m^lm is mHt.wmmmly mmmlXm Thirdp t©tr«^horia a©rr©latian 
aa@££ia4@nt,5 ara inapprQp3riat.a in ciaaaa whara gueaaing aaoura ainea t.ha 
probabilitir of a amnmm% raapanaa la influenaad not. only by latant 
ability, but chanaa faatora aa wallp raault-iner in apus'ioua guisaaing 
faatora CCarroll^ 194S> . Whlla tha aarralatlana aan bd aorraatad far 
gua^alng» othar reaaaarah <Raalcaaa» isai> haa ahown that: ava^^- or undar- 
aorraating ylalda undaalrabla raaulta* Othar problama inaluda larga 
st.andard arrara Craquiring tha uaa af vary larga aanpltia ta^ aahiav@ 
atab4Xity> and tha potantlal far nagatlva alganyaluaa <an undaalrabla 
faatura whmn alganvaluaa ara Int^arpratad aa amauat of vm^L^nam 
aaaourgtad f ar > m 

Haaant davalopmanta in .analyaia hava inaiudad tha full^ 

Infarmatian aaKimum likallhood <ML5 approaoh CBaak and Aitkin, isaii 
iaak, Gibbona, and Mtirakl, 1WB3> and the ganarali^ad laaat. aquaraa 
appraaah af Huthdn <iS7a>» Thaaa taahniquaa ara aapabla af daaling 
with diahat.omisad data and pravida a atatiatia for madal fit whlah ia 
aaymptotlaally diatrlbutad aa ahl^aquara. Howavar, tha atatlatiaal 
taat la baaad on diatributional aaaunptiona whlah may be taa 
reatriativa for tha varlablaa* Furtheraara, for teata af moderate 
mimm^ very large aamplaa ara required to Inaure tha aoauraay af the 
aaymptotio appraxiiiatlona In addltlan, raatriatiana ara plaaSd an tha 
number af itama whlah may ba factor analy^ad Caaaarding ta Hlalavy 
<l!^aS>, as la an upper limit far tha QLS praaadura> ar tha number of 
faatara in the aalution €1-3 far teata with SO Itema In the I5L 
teaUnlqua> . 

From the abava rilaauaaianp it appaara that, although aanventianal 
faatar analyala and k^GA are uaed to inveatlgate the struatura of aeta 
of itama, the teahniquaa are not without prabl^^ina when dlahatami^ed 
data are analymad. Othar praaadurea for invaatigatlng data atruature 
are neaaaaary • 



All th© teahniquss duarlbed above ara baaed on iinaar madala. in 
whi^h^ the ^mmponmmm to th© ttaiis are modaled by a weighted aum m£ 
latent, varlablea. In oontraat^ multidinensienal aaaling <MDS5 jnadela 
may be deawibed aa distance Rodela« The fundamental task of MDS ia ta 
reoover the underlying atrueture of a set of data points given the 
interpoint diatanoea^ whioh «re aasuMed to be represented by a matrix 
of ejcperinentally determined proximity measures 6ij. In MDS, neither 
the eoordinates ©f the ©bjeots beinfr «aaled ner the o©rrect number of 
dimensions <i^e,^ the aorre^t number of a©©rdinate aKes> is kn©wn« 
Pr©3€imities may be measures ©f similarity or dissimilarity , in the 
o^jse under oonsideratien herep iij is a measure of the similarity of 
itema i and 3* for example^ a oorrelation ©©effioient. 

F©r a repreaentationai spa©e in k dimensions^ a veator h is 
obtained for eaoh item whiah gives its <unknown> ©oordinatas in the 
dimensi©nal spa©e^ Then the diatan©e ©f ob^eot 1 fr©m 3 in the spa©e 
may be given by the Eu©lidean distan©el 

The Eu^idaan diatanae^ unlike s©me other distsn©e measures, is 
invariant under orthog©nal axis translati©ns. 

In general ^ dij will not equal 643 beoause ©f error in the data or 
beeauae k ia leas than the true number of dimensions in the space of 
the data^ In nonmetrl© MM, the ©eordinates of all are obtained 

subject t© a monotaniaity ©©nstraint, i.o.i. 6ij < §±^3^ implies that 
^13 i ^1*3* » In ©ther wardsp the rank orders of the distances di^ are 
essentially the same as the rank ©rders ©f the similarities 613, a much 
less restrictive ccnstraint than an equality ©©nstralnt and in general, 
representatienal apaces ©btained in nonmetric MDS will be ©f lower 
dimenaianality than in metric HDB where the equality ©©nstralnt holds. 

To measure how alasely the rank ©rders are preserved in the k- 
dimensional MDS ©©nf iguration^ a pracedure aalled mon©tane least 
squarea ar m©n©tone regression CGnanadesikan, 1S77> is carried ©ut on 
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tha dij to yield diapaglt.iaa a±3^ KruskQl haa proposed a quantity 
whiah he ealla the STOESSp &%, mm m badnaaa-©f -fit meaaur©! 



^1 



C25 



Note that the original 643 are only indireotly repraaented in Si 
through theiaf af£aat on tha rank ordara of the diaparitiaa . ym^Lmn-tm 
of this maaaure hava been proposed by Kruakal C19S4>, Takana, Young, 
and daLaauw <%977> 0 and auttman <19Sa> . 

Whila Si rapraaanta the fit of the oonf iguration in the k- 
dinanaional apaoep littla ia known about its diatributian exoapt that 
Si deeraaaaa aa k inereaaaa. Empirical raaults lad Kruakal to auggeat 
that valuea of ,aO^ *10^ .OS* .025, and O.O repraaent poarp fair, goad, 
aKcallant, and parfaet fita, raapaatively p Kruakal and Wlah <197S> 
have auggeatad that a value of •IS for k ^ 1 auggeata strongly that tha 
data are unidimensional ^ Suggoationa far ohaaaing a repreaantational 
mpmam have inoludad a aaraa-lika taahniqua examining pldta of Si va. k 
for albowa and aalaating a value for k whara a aharp band oaaurs. 
Other poaaibla taahniquaa inaluda aomparing Si va- k plata for aimilar- 
ity to aimulation raaulta far data ganeratad with a known atruotura. 

Uaina HPS with Piahotonioua Data 
Givan that MDS ia oapable of recovering tha atruotura in data 
undar a aat af leaa raatriotive aonditiana than thaaa of fmat^c^T 
analyaia, it would appear to be a patantial aalutian for tha problam of 
dinanaionality analyaia with 4iahotonoua data* Whila faotor analyaia 
requirea the uaa af aorrelation aaaffioianta or aavarianaea^ WDS anly 
raquiraa that tha aimilarity maaaurea be ordered. Given the lass 
reatriativa nature of the hypatheaea in MDS and the problems 
enaauntarad ta date in uaing faatar analysis with diohataraaua 
variablea, it is surprising that so little reaaaroh has been done on 
the use of MDS in this particular applicatian. While studies have been 
aonducted using MDS on diahotomaus data ta determine the atructure of 
the data <Karpi and Haertel 19S4| Thomas 19841 Koah 19a3> ^ the efforts 
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in t^haaa at^udlM hava bun cllr#Qt.ed t^sward mmmpmTLng jfeaulfea with 
fae-bor analysaa an real data In Interpra^ing MDS raaulta an real 
data* The true dlmenaionalitiM o£ the data mats ara unknawn^p th© 
atudiaa aannat dai^inltlvaly eatabl4ah the validity af MDS in 
determining dimenaianality , 

The Prab len a£ Di«gnaianalitv Aaaeaament 
Determining the number m£ dimenaiana underlying the atruatura of 
the data la mare aampleif than aimply determining whiah repreaentatianal 
apaae haa a small enaugh value a£ Si- Far eHamplep a twa-dimanaianal 
plat may reveal a aingle curvilinear dlmanalan CQnanadesikan 1S77> , 
Thus, the apprapriate dimenaianality may differ fram that auggested by 
the Si aoeff Icienta, Alternative teahniquea to examining valuea of B± 
have been prapaaed. One praaedure ia to aanduct a aluater analysia an 
the scaled itema using an hlerarshiaal cluaterlng algarithm ta reveal 
the data atruature. Thus, twa diatinat aluatera in a aingle dimenaion 
an an HDS plat might indiaate that twa latent variables underlie the 
data. Kruakal and Wiah ClS7d> alaa auggeated aambinlng aloaeness 
infarmatian from the MDS plata with praHimltiea Infarmatian* This aan 
be dane by aanneating all data points in the plota whoae proscimltiea 
eKaeed a aertain threahald value* This teahnlque is eapeGially uaeful 
in revealing aurvilinear atruature. 

While Guttman C196e> reaammended that these teahniques be used in 
plaae af analyaia of m% valuea, Kruskal and Wlah €lS7a> have suggested 
that the reaearaher use any neaessary praaedure ta searah for atruature 
in the data. They paint aut that the problem of dimenaianality 
asaipaament gaes beyand a aimple examinatian of B% values ar searahing 
far alusters or regions in the data. They also paint aut that the 
prablem of determining the aarreat number of dimenaians ta repreaent 
the data Ci.e*, the number af axes required far the data met 5 in an HDB 
aaaling ia not equivalent tc# determining the true dimenaianality of the 
data set. 
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Comparisan wit^h Linear Models 
Thtt m^dala far dat.a rep£'#a#nt,at,ian deacdbad by PCA and faQ^or 
analyaia are linear madala. Tha MDS modaia ara diat^anee madaia far 
whieh int.arpretat.ian af -bha aHea may diffar. In t^ha linaar madalap n 
dat.a paint^a whiah ara p^dimanaianal veQt.Qra af raapanaaa -ba t.ha i-tama 
ara pla-tt.ad in apaaa. Tha dat^a paint^a ara tha raspanaaa af ^ha 
individuaia ta tha itama and tha anaa ara tha dimanaiana deaaribad by 
tha itana« A aubapaaa in whiah tha anaa ara linaar aambinatiana of tha 
itama ia abtainad. Thaaa linear aambinatiana aanatituta tha latant 
variablaa of tha PQA^ In MOS, ^ itama are platted in a apaaa af aame 
unknawn dimenaianality and a paraimanioua rapraaantatian af that 
dimanaionality ia aaught. 

The axes of the H^M aanf iguration naad nat repreaent latant 
variablea and avan if they da, they naad nat repreaent tha aame latant 

variabiaa aa the PGA,. Thua* MDS muat be ragardad aa an alternativa 

^ ..... 

appraaah to dimanaiana! ity aaaaaamant rather than a mora general 
vara i an af PC A « 

Hante Carlo Simulati_an_ Studiea 
Ta date, only two atudiaa have been found whiah aampare tha 
af f eativeneaa of HDS with other taohniquaa far reaavering 
dlmanaionality in artif iaially^ganarated, diohotomi^ad data af known 
dimenaianality and atruature. Zwiak C19SS> auaaeaafully uaad HDS ta 
raaovar tha dimenaionality of a aat af itama with Guttman aaalin^, Sha 
auggeated that raaovary af a aingle dimanaian fram auah itema b@ used 
aa a ariterion bafara uaing any proaadura far dimenaionality 
aaaaaamant. However, ahe found that the raaulta from an HDB mnmlyBLm 
af an aatual data aat were leas alaar . 

Haakaaa <19ai> aimulatad 24 data aeta aanaiatinfr of reaponaaa of 
lOOO individuals ta SO itama « Of these data seta, 19 were 
unidimanaional , 3 had two faatora, and 1 aaah had three and nine 
faatoras Itam diffiaultiaa wera narmally or reatangularly distributed i 
and guassing parametera were aasumad aanatant far the data aet or 
normally diatributad abound eHpaatad valuaa ranging from O.O to 
He aamparad the ef f aativanaas of PCA and faator analyaia <with 
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©rthogonal and ©bliqu© salufeionsJp HDB^ cluster anaiyaia, and lt#m 
r©apons@ fehaory in racQvarAng tha ferua dimansi©nal4ty of tha data mmiLm^ 
For tha PQh and factor analya4a invastlgatlona ha uaad phi, 
tatrae^heris, and Mrraatad tatraQhori^ aorralatian ci©a£f iaienta. For 
tha MDS seallnga ha uaad 13 diffarant aimilaifity eoaffioi^nts 
Cagraaaant, approval, ata, kappa, koppa, phi, l/lmax# tatraohurie 
Caorraauad and unao«aetad for guaaainw> p Yule's Q, Yula^a V, gamma, 
tau S, and Lijphart'a indax^ . MDS tachniquaa wara affaetiva with 
aimulatad data, but ha fait that they £allad to giva an indioation of 
tha dinanaionality o£ a sat of real it©ma. 

Savaral important findlnfra amargad from tha Raokase atudy^ First, 
tha typa of aiallarity ooaffloiant uaad m££mm^md tha magnituda of tha 
valua of ais "Margin^f raa" ooaffiaianta <thoaa not affaotad by itam 
diffioulty> auoh aa tha tatraohoria oorralation and Yula's O, in 
ganaral, had largar valuaa for m± than "margin-sanaitiva" aoaffieienta 
auah aa phi and agraamant, Saoond, tha configuration of tha itama in 
tha MDS plats was m££ma%m^ by tha typa of similarity ooaffiaiant usad, 
Furtharmora, guassing tandad to dagrada tha olarity of tha plota 
bacauaa it addad random nrror to tha similarity ooaf f ieianta, affaoting 
thair rank ordarlngs. Diffloult itama wara affaotad mora than easy 
itama^ ^aokaaa raportad raaults for two-dimanaianal soalings only, so 
it waa not olaar what affaot additional dimanaiona would hava on the 
magnitudes of the Si valuaa and tha aonf Igurations of tha items in the 
ohosan rapraaantational spaaa« Furtharmora, sinoa ha waa primarily 
interaatad in tha af facta of guaasing on the plota, ha did not vary the 
strangtha of the Itam-latant dimanalon ralatlonahip within a given 
numbar of latent dimansions. 

Tha failure of the Raokaaa and Ewiek atudiaa to achieve 
intarpratable raaults for a MDS sealing of aotuai data when simulation 
rej^ults ara so promlalng la puzzling p With this in mind, this study 
was undertaken to invaatigata pasaibla raaaona for these failures and 
to detarmina tha rola of the atrangth of tha relationship of the item 
to tha latant dimanaion in tha soaling of itama, Sinoa MDS has baan 
used to Intarprat actual data, additional aimulationa are neaded to 
datarmina tha validity of this taehniqua for interpreting data. 
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The research quea^i.on@ addr^aaad in this mt.udy are, mpmaL£ i gsl X ly ^ 
Im fiwm m@t.hoda a£ dlmen@lonalit.y masessmant. using HDS affeat.lv@ In 
reaoverlng t.h@ trua dlnianalanality a£ mm-tm o£ itams which have hmmn 
dlahat.QmQualy aaared? 

2. Ar@ methada m£ dlm4inal^nallt.y ^mmmmmmmnt. ualng HDS In^luenaed 
by tphe numbas^ Qf dlmanalana undarlylng t.ha aat.a af ltama« t.hd magnlt.ude 
atf tha relatienahlp of the Iteiaa t© the latent t^alt^ the diatributlana 
a£ thaae magnitudes aeraaa the latent trslte* ar the typ© a£ aimilarlty 
neaauraaent uaedf 

3. Daea the type a£ ©Imllarity Gmm££±m±mn% used In HDS affeet the 
dealalan made in aaaaaalng dlmanaianalltyt 

4. Haw da methada of dlmenalanallty aaaeaament ualng HDS aampare 
with PCA in terma a£ reeavering the true dimenalanallty a£ a aet af 
dlchatamaualy aaared itemat Da they eliminate the preaenae af aa- 
called "difficulty" tfactara? 



Data Geneyatlan 
Data of knawn dlmenaianallty were abtalned by almulatlng 
Individual reapanaea ta a aet of hypathetlaal itema under a 
multldlmenaional eKtanalan of the twe-paramater laglatlc model <2PL) 
<Blrnbaum^ 19€a> i 

P<K ^ 1 1 a > ^ — ^ <3) 

- - -1 .7af <e ^ b, > 

i ^ ^ -1 -1 



Thia madel ia eaaentlally the aame aa the multivariate two-parameter 
logiatic madel prapoaed by Heckaae and McKlnley <19a3> who modified the 
nadel aomewhat to cape with problema af eatimatian« 

In order to replicate test conditlona approiclmately , item location 
parametera <bi> were generated ualng the atandard normal diatributian 
<for data aeta with ane^ twa« or three dimenaiona) or the unlfarm U<- 
3^^3> diatributian Cfor ane-dimenalonal data aeta> • Item 
diacrlmination parametara were specified at finad values or, in one 
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mmmm^ g@nerat.ad f^am m unifarm diatributl©n ranging be^we^n , 3 and 
2-0>. Tabla 1 presanta detail© of th© simulatad i-tem seta- Whan 
ability can b© amaumed t© have a atandard normal diatribution in the 
populatlQn and there ia no gueeaingp the relation between the item 
diasrimination parameter ai and the eerrelation aoetffiQient with the 
latent trait <the factQr loading given In Table 1) ia given by 
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<aee Lord^ 19ao> , repreaenta the aorrelation ©f the item with the 

latent trait. 

A vector a£ abilities (with length determined by the apeaified 
dimenaianallty m£ the item aet> waa generated for eaah individual. 
Abllitiea were aaaumed t© be normally and independently diatributed 
aaroaa and within indlvidualsp that ia^ not only waa the ability o£ one 
aub^eat unrelated to that of another aubjeatp but alao ability on one 
dimenaion waa aaaumed to have no relationahlp to ability on any other 
dimenaian^ 

Diahatoml^ed reaponaea ta the items were aimulated far eaoh of 
lOOO Individuala ualng <3> and oomparing P<Xi3 ^ 1 ISj) <^ ^ij5 ta a 
U<0*1) random variable Rij* If Pi^ < ^ij, Xij - 1, atherwiae Xi^ - O. 
Similarity matriaea were then generated f ram araaa-^alaaaif ioatian 
tables for reaponaea to pairs of items. Sin aaef flolents were aelected 
far thia atudy^ three margin-aenaitive Cagreement, phi, and kappa> , and 
three margin-free im/mms^^^ Yule's and the tetraahorie aorrelation> . 
Xt was felt that thaae aix aaeffiaients would give an adequate 
representation of the typea of reaulta to be eKpeated- Their formulas 
are given in Reokaae <19dl>. 

Eaoh aimilarity matrlH was acaled using nonmetrlQ HDS analyses in 
one to five dimenaiona, Euclidean distanoea were specified far 
aamparability with other analyaea and beoauae aamputatianal algorithma 
are more effiaient and mare rabuat when the Eualidean metria is uaed. 
The aealing waa aarried aut using the Kruskal algarithm in the MDS 
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prsMduif© in the SYSTAT atatistiMl paakae© CWilkinson 19a4> whioh ia 
available ^or the IBH PC. 

Wot mmmpmvLmmn with reaulta whiah would hm abtainad using 
traditi onai siothQdeXo^i^a din^n^ionaiity aasssament « PC A wad 

aa^^riad out uaing the FACTOR package in Syatat, <Thia paGkaga does not 
inalude a traditional faetor analyaia paekage, but doea permit rotation 
a£ a reduced ©at o£ principal aomponanta.) Analyaea ware carried out 
on both phi and tetraehorio oorrelation matri^^'^a. The magnitudaa of 
the aigenvaluea obtained in the analyaie were eseamined to determine tha 
appropriate number of prinoipal oomponenta to retain, 

Valuea of Si and the Si by k plots uera examined to determine 
whether they auggeated an appropriate aise for the repreaentational 
apaoa in HDS. Two^ and highar-dimenaional plota of aealed reaulta were 
eHaminad for oonf igurationa auggeating the dimanaionality of the data « 

Beauita of Ajialvm^a 

Chooaing a Repreaentational Space 

The magnitude and the behavior of the Si ooeffioient were found to 
be a funetion of the numbar of dimanaions in tha data^ the amount of 
random error in tha data, tha type of aimilarity ooafficient uaad, and, 
for margin^aenaitive aoef f isienta, the diatribution of item 
dif f icultiea. Si inoreaaed mm the amount of random error in tha data 
inoreaaad <i.e«^ aa the aum of tha aquared faator loadinga daoreaaed) , 
Furthermore, aa the amount of arror increased^ the ohanga in alopa in 
the grapha of Si vs. the numbar of dimenaiona alao deoreaaad. Thia 
made datarmining the appropriata numbar of dimanaions more diffloult. 
Figure 1 preaant© the Si value© obtained by ©oaling data ©eta 19-21 in 
one through five dimenaion© uaing an input matrix of agreement 
ooaf f ioienta . Thase data aat© are three-dimanaional with ©Imilar 
faotor atruQturei however, the aum of the aquared faotor loading© are 
•81, .25 and *0©, reapeotively . The ahange in alopa of the graph is 
abrupt at a threa-dimanaional aolution for data ©et 19, leaa abrupt for 
data ©et 20, and diffioult to determine for data set 21. The magnitude 
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o£ ^% riSAs jst.@adlly across t^heae da'ta mmt.m far m given 
dljianaienall'ty • Figura 2 r@praa@nt,a Si values Qbt.a4nad by a€al4ng 
matricaa of tatrc^harl^ eaaffiaianta £©r tha aam© data met, A mlmmr 
dlffaranca appaara batwaan the graphs £©r data aeta 19 and 20 and that 
for data aat 21s Thia graph waa aomawhat unuaual rf'ar tw© reasana, Tha 
nagnituda ©f Si in»aaaed allghtly with ln«aaaing dimenalanality in 
tha raprasentatlanal apa^e for data aet 19. Thia affaat w^a 
anoQuntarad oQsaaionally in aituatiana whare tha magnitude af Si waa 
law and f/imm aanaldarad af no importan^a. Saoondlyp the magnitude far 
Si waa appraMimataly tha aama far data aat 19 aa for data aet 20, The 
ana alaar differanaa ia tha lack of an elbaw at a two-dimanaional 
aalutian far data aat 20. CanQluaiona regarding the number of 
dimenaiana in the repraaantatianal apaoa be diffiault to make far data 
aat 21* Furtharmara^ tha numbar of d;Lmenaiana would diffaar aecarding 
to whathar tha agreamant ©r the tatr^iohori© aaeffiaiant waa being uaed. 

Far unidimanai^nal data aats 1-9 a ana-dimenaianal 
rapreaentational apaae gave law valuaa of Si t< mlBl when margin- 
aenaitiva aaeffiaiant^ were mamlm^m Am the amount of random error in 
tha data inareaaadp Si inareaaad. For the margin^free aaef f ioienta, Si 
waa vary high <.4-.5> for a ana-dimanaianal aalutian. It tended to 
drop aff ataadily with no aign af an elbaw, Thia preeludaa the uae af 
a aritarian value auah aa .15 far datarmining the appropriate numbar of 
dimenaiana in the repreaentatianal apaae when margin^free aaaffiaienta 
ara uaed. Hawavar* auah a aritarian might b© uaeful if margin- 
^enaitiva caefflaianta were aaalad. 

Far tha twa-dimenaianal data^ an examination af Si for margin-- 
aenaitive Qaeffiaianta auggeatad a two^dimenaional repreaentational 
apaae ^ However » it waa more diffiault to determine tha appropriate 
dimanaionality far data aeta with aubatantial random error (e.g.* 12). 
In thaae data aatap Si valuaa ware high and na elbow wa^ alaar in the 
graph. Margin^frae Qoeffiaienta generally had law valuea af Si for a 
ane-dimenaional repreaantational apaae, although the valuaa again « waa 
affeatad by tha amount af randam arror. In data aata 15^18 ^ tha Si 
aaaffioiant behaved aa if the data were unidimensional • 



13 

16 

EKLC 



Thm t.hrae-dinenaidnal dm-tm yielded molu^iana which ware aimilmr 
f©r t.h@ tpWa-dimenaienal datpM ^icaept. t^hat. one additional dimension waa 
nm^eaaary in ^hm aolu^ian* Thia auggaat.3 -bhat when a data mat ham k 
latent, traits^ k dimanaiona artf requirad far S'.ialing maE^gln-aenaitive 
aoa^ficienta and <k^l> dimenaiQna m^m raquirad fasf max'gin^f sree 
QQaiCf iaienta # 

Of tha margin-aanaitiva aimilarity coaffiaienta used, the 
agreeiiant aoai'ficiant had the laweat Si valuea and wae the leaat 
aenaitive to random arx'or* Whan aaardinatea were anainined fer itema 
acaled uain# a matrix a£ agreamant aae££iaients^ it waa alear that ana 
dimanaian in the repreaentatianal apace waa eimply item di££iaulty« 
Whila the kappa and phi aaaff iaiente were alao aensitive to item 
di££iaulty, the ©ffeat waa leaa atrang, ainae the effect af item 
di££iaulty on theaa aaeffiaianta ia a raatriation of range « While it 
waa nat diffiault in moat oaaea ta determine mn appropriate number af 
dimanaiana in the repreaentational apaae uaing the agreement 
aaeffiaientp it waa aanaiderably more difficult uaing the kapp^ and phi 
aaef f icienta # Far unidimenaianal data, agreement aoeffiaienta ranged 
hetwaen .02 <fQr data aet 4> and .29 Cdata aet S> . 

Data aeta with rectangularly diatributed item difficulties tended 
to have lawer Si valuea than their narmally diatributed counterparta • 
This ia prabably bacauae the itema are unifarmly diatributed along the 
aaale of difficulty. Am a reault^ they are more apread out^ making 
reveraala in rank ardering leaa likely » 

Althaugh the tetraaharic aorrelatian aaefflaient tended to yield 
Si valuea which were aimilar to thaae far Yule* a Q» it aacaaianally 
yielded much amaller valuea when factor loadinga were high. 
Examination of acale valuea and item plota indicated that a difficulty 
dimenaian waa preaent far the tetraaharia correlation. Thia waa nat an 
artifact af the acaling praaedurep ainae the aame effect waa noted when 
PGA w<^i|t- carried out an the matrix af tetrachoric correlmtiana . 

Na aimple reaommendatione can be made regarding chaice af an 
apprapriata repreaentatianal apaae from theae reaulta. When a aharp 
elbow ia preaent in the plot of Si by dimenaianap a choice af the 
apprapriata number af dinenaiQna ia clear. If the initial level of 
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H^wQVdrp aa will b@ aeen £rem an ^^ainlnat.i©n of item plota, %hm 
inta^ptatatian of thla spaQa dij'fara aomewhat. batwa^n margin-aenaitiva 
and margin-£raa €@af fiaiant^a* When n© low valua ©£ a^raaa la reaQhed 
for a raamonabla <aa dafinad by tha s^aaear^har) numbar of dimenmiona, 
itam plot.a will hava to hm aHaminad to determina whaUhar thay aaat 
addit^ional light on tha iaaua. 

Raaulta of E^aminat.ion of Eiganvalu^^ 

FOA tandad to yiald good raaulta with tha aimulated data* Thia 
was axpaetad, ainaa tha data ware ganaratad uaing a modal for whioh PC A 
should ba maximally Qffeotive. Thia might not be the oaae for data 
with a nonlinear undar lying models 

Tha magnitudas of the aiganvaluaa wara affa^tad by the numbar of 
dimanaiona in tha data, tha amount of random error preaent in tha data, 
the dlffloulty level of the itema, and, when an item loaded on multiple 
faatora, the diaorapanoiaa between those loadinga. EKoept in tha oaaaa 
of data aeta eontaining fawar faotor atruoturaa than faetors Cfor 
aKampla^ data aata lS-ia>, PCA yielded aolutiona with aa many or mora 
large aiganvaluaa than faotora. Spurioua faatora due to diffloulty 
were noted for analyaaa of both phi and tatraohoric eorrelation ooaffi- 
oiantai thaae were more likely to ooour whan oorralationa with the 
latant faator were high. The tatraohorie matrioaa in auoh inatanoaa 
tandad to be non^Qramian and tha eiganvaluea repraaanting aystamatie 
effaeta wara aubatantially higher than thoaa obtained in analysing tha 
Gorreaponding matrix of phi ooeff iaienta. Figure 3 preaanta graphs of 
eiganvalues obtained in PCA of three-dimenaional data aeta 19-21 uaing 
phi eoef f iolents. Figure 4 presents reaults of analyses on the same 
data aeta using tetraahorio oorrelationa. Am the amount of random 
error inaraaaed in tha data, the magnitudes of the eiganvaluea of the 
firat three prinoipal oomponents deoreasad. Sinea tha phi ooeffiaienta 
tended to yield lower eigenvalues for tha primary prinoipal components 
than the tatraohorie eoef f ioienta, result, the elbows in the graph 
appoarad leas wall-'def ined« 
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Tw© addlt.l€inal thins© were faund affeet the magnitude of 
eigan values aorreapanding to principal eompanenta regardless a£ whether 
phi or tetra^horie earrelationa were analysed. First o£ all, the 
nagnitudes are affected the heteregeneity of the earrelations with 
the latent trait in multidimenmianal sets. In data sets 10-12 and 19- 
21, eaoh item loada ly ©ne of the k < ^ 2 ar 3> tfaatars in the 

item eet and the £irst k prinQipal aamponents are appraHimately equal. 
In data sets 13-14 and 22-25, as the k £aator laadings become mare 
hamageneaua, a single principal aampanent tends to daminate the 
eiganvaluea. Secandly, the magnitudes af the principal aampanents are 
affected by the heterogeneity af the items. Data sets IS-ia are very 
aimilar to data seta 13-lS eKcept that all items have the same factor 
laadinga an eaah factor. Whereas data sets 13-lS had a daminant faator 
and a smaller secand factar, data sets IS'-ia had enly ane daminant 
factor* These findings have twa implicatians. Firsts the principal 
campanents need not carrespond ta the original generating latent 
tsraita. This hae implicatians far interpretatian af principal 
campanents. Secand« the dimensicnality af an item is a functian af the 
set af items within whiah it accure. This is an example of B^jar^s 
<19d33 statement that dimensianality , like reliataiiity* is situation 
specific* Clearly^ this hae important implicatians for teat equating* 

While data sets 13-15 appeared to have a single daminant factor 
and a subatantislly smaller second factor Cacaarding ta eigenvalue 
analysis) ^ many researchers would declare the set af items ta be 
unidimensianal . On the ather hand^ Si by dimensian plats clearly 
ahawed that these items were behaving in a similar fashion ta data sets 

It may be, then, that HDS, because it does nat attempt to form 
linear cambinatians, is better at recognising dimensianallty * Si by 
dimension plots for data sets 18^18, however^ behaved like thase for 
ynidimansional data sets* The reason for this will become clear in the 
ne^t section* 

Isiamlnation af MDS Plats 

Items are lacated in the MDS repreaentational space acaarding ta 
their similarity. The items with the highest similarity caefficients 
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will be the items with th© meat similar faetor loading©. Am m result, 
items with aimilar faatdr los^inga tend to oluater together. This 
effeet was apparent in all data sets regardless of the type of 
aiimilarity Qoefflaient used. However^ the margin-sensitive 
aoe££'ioients eontained an extra dimension due to dlffioulty whiah 
pulled the clusters apart giving a linear appesranee* The appearance 
m£ the line varied with the type of margin-senaitive coefficient input 
into the scaling algorithm. Since the scaling results are essentially 
the rmmm except for differencea in appearance caused by the difficulty 
dimension, results will be diseuaaed for the margin-free coefficients 
first. Data sets 1-9 all yielded a single cluster or disk of items. 
As the amount ©f random error increased^ the disk tended to pull apart, 
but it was still recognisable as a single structure. Data sets 7-9 
contained items with different factor loadings on the single factor. 
Figure S presents results obtained for data set a for a scaling of 
Vule^s Q coeff ioienta* Items labeled 1^ 2, and 3 correlated .9^ ,8, 
and .3 with the latent trait, respectively. When the items in theme 
data sets were scaled, the items most closely related to the latent 
trait appeared on the Interior of the cluster in a small region. As 
the relationship of the items to the latent trait weakened, the items 
were moved away from the center of the cluster and spaced in concentric 
aircles about the center. This suggests that the magnitude of the 
factor loading controls the tightness of the clustering. 

As the number of dimensions in the item set increased, the number 
of clusters increasfed. Data sets 10-15 yielded two clusters of items; 
data sets 19-24 yielded three clusters of items. Figure S shows 
scaling results obtained for a matrix of tetrachoric coefficients for 
data set 19 which had factor loadings of .9 on one of three factors p 
Items related to the first, second, or third factor are identified with 
a 1^ 2, or 3. Three clusters are clearly visible here (although two 
items trail from the clusters) , 

As noted previously with the unidimensional data, the tightness 
the clusters decreased as the amount of random error in the data 
increased. Examination of the item plots suggested why the one- 
dimensional HDS solutions for the margin-free coefficients had such 
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high Si values. When tw©- and h±gher^4imenaiMal data ar© aea;.ad, 
^l££mTmnamm ameng itama due to random error are amall compared to 
di^££mrmnamm due to asaoaiation with different oombinatlona of faatora. 
Am a reaultp rank order In^a o£ iteii diatancea are. likely to oontain 
£ewer large differenoea in rank orderinga resulting in a lower Si 
valuer When ona-dimenaional data are m^mlmC^ differeneea among itams^ 
are am^ummd. only by random errors Rank orderinga are random and reault 
±n a high Si value« For n obje^ta, n-1 dimenaiona are required to 
aooount for the differen^aa in a aatiafaotory manner « 

When the aorrelationa are low, there are aubatantial amounta o£ 
random error to aoaount far* and* aooordingly, th^ae itama ar© soaled 
farther fro&i the aluater center* For the multidirtienaional data aetm, 
the differentiation between the aluatera tended to deoreaae aa th© 
amaunt of random error inareaaed. Thia ia beaauae the ayatamatie error 
aomponent Cthe aemponent whiah influenaea the aeparation of th© aluater 
aentera> beaomea amall relative to the random error aomponant <the 
aomponent whiah influenaea the diatanae of the itema from the aluater 
aentera> . Aa a reault* the leaa reliable itema tend to overlap- 
When the three-dimenaional data war© aaaled* aluatera were loeatad 
auah that itema with identiaal loadings <but on different faators* mm 
in data aeta 1S-21> were equally distant from ©aoh other* Whereas thla 
oan be don© on a lii^e for two-faator data* an equilateral tri^mgl© is 
required for thr©a-f a^^tor data and the repreaentational apaa© will be 
required to have two dimenaiona* Thia suggests that a k-dimensional 
latent spaa© will require €k-15 dimenaiona in an MDS apaoe Cunlesa a 
^-idff i^ui'fcy dimenaion is required^ . The vertiaea of the triangle may be 
aonsidered to represent the aimplast faatora. For example * itema 1-13 
in data set IS would form a aluater at one vertex* items 14-27 would 
form another aluater at a aeaond vertex » and the remaining itemm would 
form the third aluater at the last verte^c. Th© number of distinat 
faator struaturea in the data aontrols the number of aluat©rs. For 
thia reason* data sets 13-15 will form two alusters ev©n though th© 
items are highly related to eaah other. However* their separation will 
be Gontrolled by the relative atrengtha of their loadings on the latent 
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vaE^lableai th© differentiation of tha aluatera will be c©ntreill©d hy 
tha magnitudam of the loadings on the latent vas^iablea. 

Data sets 27-29 provide an illustration of the above point. Two^ 
dimensional HDB plots are presented for these data in Figures In 
data set 27, thriPie aluaters are distinatly visible* Items whioh are 
related to the first faator < labeled 1> sre located in a elueter on the 
left of the diagrami iteme related to the seoond factor oluster 
< labeled 3> on the right. Iteae whioh are related to both faotors 
< labeled 2) are looated between the two single-f aotor alustera* Sinee 
they are equally related to both faetorsp they are spaoed midway along 
the line joining the oenters o^ the two single-f aator olusters. Items 
1-13 and 2©-40 are highly related to their respeotive faotors and their 
dispereal from the cluster oentar is small. The two latent faetora 
aacount for only 50^ of the systematio variance in itema 14-27 <as 
opposed to BIH in the remaining items> , so the oluater appears more 
diffuse. In data set 26, items 1-13 Clabeled 1> were more heavily 
related to the first faetor than the second €.7, .33, items 14-28 
Clabeled 2> were related equally to both faetors C.S, .5) and items 28- 
40 < labeled 3> were more heavily related to the second factor than the 
first <*3, *7> . The resulta in Figure 8 show that, again, three 
clueters were formed with the items equally related to both factors 
located toward the center of the plot* However, the clusters on the 
left and right aides of the plot were more spread out than in data set 
27 where factor loadings accounted for B%H of the variability. In data 
set 28, the faetora only aooount for BBH of the variability in the 
first and third clustera. Differentiation among the cluatera waa 
aaoordingly lesa clear. 

Data set 29 yielded the most informative results , The plot 
obtained by scaling tetrachoric correlations for this data set im ahown 
in Figure 9. Here there are aeven distinot factor structures, with 
items loading on one, two, or three faetora. Single factor items were 
aaaled in clustera according to their factor relatJ.onship (i.e., items 
^^^p 7-12, and 13-18 were scaled in three separate cluatera labeled 1, 
2, or 3> . These , cluatera were equally apaced as in data aeta 19-21. 
The cluaters containing homogeneoua sets of items which loaded on two 
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oaf the three £actQrs Clabeled 4, 5, or 6> ware loaated near a line 
drawn between the two verti^aa the triangle containing the factcra 
to whioh the itema were related. Thua^ if a aluater contained itema 
related to faotora 1 and 2p it was loaated between the oluater for 
factor 1 and that for factor 2* The eluater waa loeated cloaeat to the 
oluater to which it waa moat highly related. Therefore, a Qluater 
containing itema generated from €-7, .3, *OD factor loadinga waa 
located between the clustera generated by <m3, ,0> and <.0, ,9, .0> 

faotor loadinga^ but clcaer to the former than the latter. The cluatar 
containing itema generated by .5^ sSJ factor loadinga <lataeled 7> 

waa located at the interior of the triangle. Aa itema take on a more 
complex factor atructure* they will be pulled away from a line or a 
point toward other factora on which they load. Thia auggeata th^t a 
tetrahedron in three dimenaiona will be required to repreaent a four-^ 
factor atructure and that itema will be located in the interior of the 
tetrahedron if they relate to all four factora. 

Theae findinga have important implicatlona for aaaeaament of 
dimenaianality in MM. An examination of the cluatering in HDS plota 
will help to iaolate aeta of homogeneoua itema. However, if the goal 
of the HDS procedure la to determine the number of latent dimenalona 
preaent in the data* the magnitude and behavior of the S% coefficient 
nuat alao be conaidered. 

Plota for the margln^aenaltive eoeff icienta mppmuL^^d quite 
different becauae of thta preaenca of a dimenaion related to difficulty, 
yhen unidimenaional item aeta were mmmlmd. uaing agreement, kappa, or 
phi coeff icienta, itema were arranged in a line with the moat difficult 
itema at jone end and the leaat difficult at the other. For the 
agreement Qoefficient, the line waa atralght. However, for the kappa 
and phi coeff icienta, the line waa in the ahape of a horaeahoe. An 
eHample of the type of plat obtained ia ahown in Figure lO, which ahowa 
the reaulta of acaling kappa co^ff icienta in data aet 1 . Itema ar@ 
ordered by diffiaulty with A the eaaieat Item, pragreaaing through the 
alphabet to numbera and aymbala. Nate that while two dimensiona are 
required to repreaent the graph aatiaf actorily , a aingle curvilinear 
dimenaian could be hypathealsed for the data. The kappa oaef ficient 
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was spMitflcally suggastad to daal with tha problems of ohanea 
agraaaant. dua to jRa^ginal dlatributlanap so it ahauld be Immm aanaitlva 
to i.tam difficulty than the agraement ooaffioient* However , both kappa 
and phi have rangea which are reatrloted by dlf f arena^ff^ in item 
^i^^i^uitieai hanaa^ their valuaa are atill affected by item 
difficulty p Thia ia eapacially true at the extremes of the 
distribution and may help to account for the curvilinear effect. 

Am the correlation af aaah item with the latent trait decreased, 
the alarity of the plota tended ta decrease. The phi aaeffiaient was 
aomewhat mare affeated by thia prablem than kappa* Am the random error 
increased^ items tended to vary from the line or haraeahoe degrading 
the alarity of the original shape* Plots using kappa aoefflaienta 
tended ta take an curvilinear farms other than a harseshoe, while plots 
using agreement coefficients tended to reaemble a fllled-in ellipse . 
This inareaaa in scatter about the "true" item loaation has already 
been deacribed for cluater-type plats. 

yh^n data contained two or more dimensions^ plots using agreement 
mGtm££ialmn'tm took on a circular or elliptical ahape. Figure 11 
preaenta resulta far acaling a matrix of agreement coefficients for 
data set 19. Items are labeled accarding ta th© factor on which they 
load moat highly, as can be seenj^ items are aaaled in area about an 
axis. Examination of the item difficulties showed that the anis was 
related to item difficulty with easy items scaled at the one end and 
^^^^Acult itema at the other. Items which were of extreme difficulty 
or eaainaaa were closer together than items of moderate difficulty. It 
appeara that auch itema are more like eaah other because af their 
difficulty ar eaainess than beaauae af any relationship to a factor. 
However, since an item related to one factor must be aa distant from an 
item of the same diffiaulty related to another factor as it is from any 
other item of the aame diffiaulty related ta a third factor additional 
dimenaiana will be needed ta represent the distances in a aatisfactary 
manner. Far three-dimensional item sets this structure was similar ta 
a football. 

Plots for kappa and phi tended to resemble the plots for the 
margin^free aoeffiaients as mare dimensiana were added to the data with 
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it©ma ordered within the elusters aaeording to dl^fieulty. In general ^ 
kappa, phi* and ^/^mmu tended ta have plots whiah were most influenced 
by random error in the data and to yield the moat variable reaults. 
For this Tmmmmn^ their umm is not suggested « Results for Vule^s Q and 
the tetraohoriQ ooeffiaients were very eimilar and either of the two 
would yield good results. Ixoept for oeeasional problems in values for 
the tetrschorlo correlation with a relationship to item difficulty when 
the Gorreiatian with the latent trait was very high, either oould be 
uaed interchangeably. As previously noted, the agreement coefficient 
is suggeeted when examining data for unidimeneionality , since only the 
margin-sensitive ooefficienta were Qapable of indicating a one- 
dimensional solution for suoh data. 

Cluster Analyaem 

The olustera present in the HDB plote suggested that cluster 
analysis might be appropriate for classifying items i^to hamogeneous 
sets. While Heckase <isai> did investigate cluster analysis as a 
technique for identifying homagenecus Item sets* he applied it directly 
to the same similarity matrices used in the MDS. He found that many 
items were misclassif ied even when the carrect number of dlmensians <or 
olustersj was used. Furthermore, he found that margin-sensitive 
aoefficients tended to group items of similar difficulty* 

Although cluster analysis has a number of problems as a 
elassif loation technique, two are especially difficult for the 
P^^^'^it-i^ner - susceptibility to local solutions <in some cases, due to 
the presence of outliers) and lack of measures for determining the 
optimal number of clusters in the solution. However, since the results 
were clearly clustered, the technique was applied to the coordinates of 
the items in the HDS representational space <rather than the similarity 
matrices based on item data> . It was hoped that the technique would 
permit identification of^ homogeneous item groupings even if it did not 
give information about the number of dimensions in the data. In 
addition, using the HDS coordinates might reduce the sensitivity to 
outliers. The first two cluster analysis procedures used were single 
linkage and complete linkage options in the CLUSTER procedure in the 
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SFSS-X atatistiaal pa^kmge CSPSS-X Uaer^s Guide, ItaS) . Euclidean 
dlatana© mmmmwmm wa^e uaed with th© data. It warn hoped that ana of 
thaae proe©durea would hm auQCMSful with tha MDS raaulta ©bt«inad from 
the agreement matriQee* However, neither of theae tMhniquea proved 
uaeful in almmm^£y^ng itema into eluatera. Glaaaif ioation appeared to 
toe random and theae teehniquea were dropped* The asaling eoordinatea 
obtained for matrices of tetraehorio aoeff ieienta for data aeta 27-29 
<two- and f ive-dimenaional aolutiana> were input to the k-meana 
eluatering program BMDFKM CDixon, 19a5> , This technique ia more 
effective with the apheriaal-type aluatera obtained in the HDS analyaea 
and gives information on the within-aluate^ and between-aluater mean 
aquarea for each variable <here, HDS dlmenaiona> to permit aeaeaament 
of the relative importanae of eaah variable in alaaaifying the data. 

In general^ reaulta of the k-meana aluatering were good* No oaaea 
were miaalaaaif led when three aluatera were apeaified for either two- 
or five-aoordinate HDS aolutiona in dsta aet 27- When more aluaterm 
were apeaified than were aatually needed, the algorithm aimply aplit 
the larger center cluater into mmmllm^ onea. When fewer aluatera were 
apeaified than the correat number, the aenter aluater waa split so that 
half the itema which loaded equally on both faatora were aaaigned to 
each of the aingle-f actor aluatera* Significance testa on the 
importance of eaah variable to alaaaif iaation revealed that only the 
firat dimenaion waa aonaistently uaeful in aiaaaifylng itema into 
groupa- Am the nufiber of aluatera requeated increased, additional 
dimensiona were required to aeparate the itema into homogeneoua groupa, 
but they were of subatantially leas Importance to the alaaaif ication 
procedure * 

More problems were encountered with cluater mnmlymmm in data set 
2a which had lesa differentiation between the cluatera, initial 
analyaea of both two- and five-dimensional data miaclasaif led about 
one-third of the cases, primarily because alustera 2 and 3 were aplit 
on a horiMntal rather than vertical a^ia. However, when eeed valuea 
were specified which appeared to be near the clumter amntmvm, no cases 
were mlaalasaif led for the twa-dimenalonal data and only one for the 
five-dimensional data. h similar problem waa encountered for the 
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a^v@n^€ilus^@r salut.lQn for data s©t 29. AlthQugh all mmmmm were 
Qlaaai^lad correQt.ly when the tWQ-dlmenalonal sealing reault.a were 
input p the f Ive-^dimenaional data gave poor reaulta. Kewever, after 
seed valuea were apaeifiedp only tw^ saaea were miaclaaai^ied. Thia 
auggeata atrongly that the r^'^^areher aheuld try multiple aQlutiana 
with different aeed vaiuesp particularly in regiona where Qluatera are 
auapecrted. If reaulta are atable f^r a number m£ sslutiona and i£ the 
aolutiona aeem reaaonable^ then the eluater a@luti©n may be accepted. 
Theae reaulta ala^ auggeat that more dimenaiona than neceaaary far the 
HDS repreaentatianal apaea ah@uld net be input into the Qluatering 
prQQedure « 

One additional problem encountered in the analyaea waa the laok of 
a criterion for determining the optimal number of oluatera. Sinoe the 
plotting reaulta clearly auggeated the correct number of ciuatera* the 
problem waa not difficult here. The C-index CSeber 19a4> yielded poor > 
reaulta and failed to indicate the correct number of cluatera^ When 
the atatiatic waa modified to include only thoae variables which were 
aignif leant in the cluateringp the reaulta were very good« More 
reaearoh needa to be undertaken to determine what, if any^ indicea of 
cluatering may be uaef ul , 

Gonql um ion a 

Aa in the Reckaae Clsai) atudy, MDS haa been demonatrated to be a 
uaeful technique in claaaifying itema into homogeneoua groups when the 
aourcea of variability in the data are well^apecif led. While Reckaae 
waa intereated in the effect of gueaaing on HDB reaulta , thia atudy haa 
focuaed on the effect of the relationahip of the item with the latent 
variables Although aome differencea have been found which may be due 
to differencea in acaling algorithmap reaulta are in aubatantial 
agreement. Heckaae found that gueaaing tended to degrade the quality 
of the item plota* In thia atudy ^ it waa obaerved that mm item 
Gorrelationa with the latent trait decreaaedp itema were more widely 
diaparaed about the cluater center. In both mmmmm^ the dlaperaion may 
be attributed to increaaed random error in the datai for the Reckaae 
atudy p difficult itema were moat affected p whereaa in the preaent 
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atpUdy« all items were affeeted regardless ©£ their diffieulty level. 
Although Heakaae had one data set whlGh aontained more factor 
structures than £aat©rs Cand whiGh yielded aa many eluaters mm 
structurea> ^ for the mast part, the two variables were aonfounded^ 
Thia study ah©ws alearly that the number of olusters la related to the 
number of distinat faetor structures *n the data* The reaults of this 
study auggest that the number of factors in the data is related to the 
number of dimenalons in the representational apaoe* but that factors 
and dimanaiona must be dlatinguiahed in MDS. 

Of the GQefflclents uaedp three CkappSp phlp and 5/5,|iax> were 
found to be partlaularly aensitive to random error and their uae is not 
auggested. Of those remaining, the agreement eoefficient was found to 
be useful in caaea where unidimensionality is eKpeated in the data, 
While it is aenaitlve to Item difflcuityp thia dimension was clear on 
the item plots. The tetrachoric aoefficient and Yule^s Q are 
Partlaularly uaeful when the goal of the analysis la to isolate 
homogeneous aeta of Itema ainae they tended to yield cluatered reaults. 

Because of the tendency of itema to cluater together according to 
their similarity^ it la auggeated that cluater analyais of the MDS 
coordinates be conducted to isolate homogeneoua aets of items. While 
the tendency of the technique to yield aolutiona which are aenaitlve to 
local problems should not be overlooked p some of the problema may be 
rectified by using seed values for aluster centers. Thia has the 
advantage of permitting the researcher some aontrol over the aolution. 
Becauae misclaaaif iaationa tended to oaaur when MDS repreaentational 
apaaes had too many dimensional it is suggested that no more than the 
neaeasary number of dimensions be uaed for the analyaea. If two- 
dimenaional HDS reaults are adequate to determine the structure of the 
datap then no more than two dimenaiona should be usedg 

These results deal only with simulated data. Further reaearch 
ahould be carried out to determine whether these resulta are 
appropriate for actual test data. In addition * reaearah needa to be 
carried out to determine whether any of the measurea of aluster Ing is 
uaeful in determining the optimal number of alustera in this aitustion- 
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Cap^lQna for Flguram 



Figura 1. Si by dimanalon plot for three-dlmensifjnal data aats 
19'-21 Cagra©mant amm££LaXmnt^ • 

Figuaf© 2- Si by dimenaian plat £ar thrae-dimansional data aata 
19-21 <t©tra^h©rla ea©f£±eient> , 

Figure 3. Eigenvalue magnltuda far principal aampanenta 
analyais a£ data aeta 19-21 <ph± Me££±alenta) . 

Figure 4. Eigenvalue magnitude far principal TOmpanenta 
analyaia af data aeta 19-^21 Ctetraeharia eaef f ieianta) . 

Figure S. MM reaulta far ane-^dlmenaianal data aet CB> , 13 
itema ,9, 14 itama 13 itema ,3 CYula^a Q> , 

Figure 6« HPS Heaulta far thraa-dimenalanal data set €19> . 13 
itama <*9^ 0> , 14 itema <0, .9, 03, 13 itema <0, O, .9> 

<tetraaharia aaeff iQient> • 

Figure 7* MDS Heaulta far twa-dimenaianal data aet <27> , 13 
itema <*9, 0> , 14 items C.S, .S) ^ 13 items CO, .9) Ctetraaharia 
aaaff iaient> • 

Figure a. MDS Reaulta far twa-dimenaianal data aet <2B> , 13 
itema <*7^ .3>, 14 itema <,5, *S>, 13 itema <.3^ t7> 
Ctetraaharia aaaf f ielent> « 

Figure 9- MDS reaulta far three-dimenaianal data aet C29> . 
Kixed faatar loadings ctetraaharia aaeff iaient) . 

Figure 10, MDS reaulta far ane-dimenaianal data aet, ne^mally 
dlatrAbuted item diff laulty Ckappa aaeff laient) , 

Figure 11- MDS reaulta far three-dimenaianal data aet C19> 
€ agreement aaeff iaients . 
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